Assumption-Free Exact Matching Inference in a Large Population of Data
2025-07-30
Managers fear that the preferential hiring of military veterans leads to the selection of employees who score worse on merit criteria. Using all non-classified records in the U.S. federal government’s Central Personnel Data File from 1973-1997, we compare the educational backgrounds of employees who received veterans’ preference benefits with the educational backgrounds of employees who did not receive veterans’ preference.
The design:
We replicate prior evidence and media accounts that preference beneficiaries possess weaker educational credentials than non-recipients.
Then we make this relationship disappear—and, in some instances, invert—when we account for employees’ occupations and work circumstances using exact stratification.
Takeaway: The assessment of perceptions of hiring quality requires careful attention to the appropriate basis for comparing employees.
van Riper’s History of the United States Civil Service notes it is one of the oldest personnel policies in the federal government.
Veterans’ preference predates the competitive civil service exam according to the Merit Systems Protection Board.
Lewis (2012): Under the policy, eligible military veterans—as well as the spouses and mothers of veterans who were disabled or killed in combat—receive either added points on their federal service applications or hiring priority over candidates categorized in the same quality tier.
Quite frankly, we have both positive and normative motivations that we should lay bare. The topic is of both public and academic interest.1
In various forms for almost a century, Miller (1935), Ordway (1945), Lewis (2012) maintain that these procedures force managers to distribute jobs to preference recipients, even if other candidates appear to be more deserving of a position
Lewis 2012: 247: ``Federal personnel data for the past decade show that veteran new hires are older and less educated than nonveteran new hires, and that they do not advance as far in the first 15 years of their careers as nonveterans hired into the same grades at the same time, suggesting that veterans’ preference may be lowering the performance of the federal civil service.’’
Elsewhere, Johnson and Walker (2018) uses pay grade and pay increase data to show equivalent advancement tendencies for preference recipients and non-recipients when entry positions are rendered comparable.
Unlike the 1% samples, in the summer of 2009, the National Archives and Records Administration (NARA) released a copy of the U.S. Office of Personnel Management’s (OPM) Central Personnel Data File (CPDF) under a FOIA filed by Tim Johnson. This copy of the CPDF contained all non-sensitive employee records from 1973 to 1997. The data are a person-year panel with 3.7 million individual persons.
Past scholarship has claimed that veterans’ preference lowers the educational attainment of the federal service by preventing the acquisition of better-educated personnel.
This yields 3.8 million employees in their first year of service.1
Figure 1
Table 2
Table 1 Excerpt
Our counterfactual of interest, defined by the job as opposed to the person, is, ``had veterans’ preference not played a role in installing the job’s incumbent, how would educational attainment differ for the holder of that job?’’
With caveats, when we retain all data, the conventional wisdom mostly holds though 10 point preference recipients are much higher in 1979 and 1980.
But… year doesn’t seem like the best descriptor of a job.
Wilcoxon statistics for Occupations
The top two results have switched signs: 5-point and 10-point Veterans’ Preference recipients have higher levels of education; the evidence for spouse/mother is a bit mixed.
Education by Preference Status for Clerk
Note the abundance of white on the right side of the figures. Higher education levels for preference recipients.
Most stringent job and education
Notes: The y-axis is censored at 200. The result from this is the top row of the next table.
Prior evidence has led to what Pearl (2012, 176) calls a distorted causal interpretation. The broad universal association is quite different than the conditional associations when the job becomes the counterfactual. Though it is claimed that any deviations from merit screening permit less-qualified individuals to enter the federal service, merit screening is only one of many filters.
On balance, we find very little evidence that veterans’ preference worsens the educational attainment of job incumbents.
How could we account for the finding that veterans receiving preference appear to possess equal or greater education in many cases? The GI Bill comes to mind.
Ala Pearl’s discussion of Simpson’s Paradox, given that veterans’ preference accrues to veterans that have access to educational benefits, it is quite easy to see how a relationship in the aggregate showing lower levels of education among preference recipients should fail to replicate for at least some jobs if veterans distribute themselves across occupations differently than nonveterans and veteran status correlates with potential educational benefits.
A DEI policy that has no discernible impacts on key elements of expected job performance. Combined with evidence about career trajectories, the belief that DEI policies lead to less qualified candidates is, in this case, quite mistaken.
There is little evidence, given reasonable definitions of a job, that incumbents with veterans’ preference had lower educational attainment than those without preference; the reverse is more plausible.
Distributions
Prompt
Results
Veterans Preference in the CPDF